What  Happened  to  DT&E? 


Steve  Hutchison,  Ph.D. 


The  office  now  responsible  for  overseeing  developnnental  test  and  evaluation  (DT&E)  was 
created  four  decades  ago  to  oversee  all  test  and  evaluation  (T&E)  in  the  Department 
of  Defense  (DoD).  In  the  January-February  2014  issue  of  Defense  AT&.L  magazine,  I 
described  David  Packard's  response  to  the  Blue  Ribbon  Defense  Panel  in  shaping  the 
original  office  in  the  Office  of  the  Secretary  of  Defense  (OSD)  responsible  for  T&E  over¬ 
sight.  In  this  article,  I  describe  the  DoD's  efforts  over  the  past  40  years  to  shape  T&E  oversight 
organizations  to  help  improve  acquisition  outcomes.  Ultimately,  this  article  is  intended  to  provoke 
a  rethinking  of  how  we,  as  testers  and  as  members  of  the  acquisition  community,  can  better  help 
programs  provide  enhanced  capabilities  to  our  warfighters  in  an  effective  and  timely  manner.  If 
that  is  not  our  top  priority,  then  I  think  we  may  be  in  the  wrong  business. 


Hutchison  previously  served  as  the  principal  deputy  for  developmental  test  and  evaluation  in  the  Office  of  the  Secretary  of  Defense. 
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The  key  to  improving  acquisition  outcomes  is  to  properly  set 
the  conditions  to  begin  production.  Said  another  way,  im¬ 
proving  outcomes  is  not  about  increasing  the  pass  rate  for 
initial  operational  test  and  evaluation  (lOT&E)  or  the  number 
of  programs  that  get  to  full-rate  production  (FRP),  because 
those  numbers  can  be  very  high  yet  require  significant  post¬ 
production  costs  to  repair  or  add  capability  we  wanted  but 
didn't  get  at  initial  operational  capability.  Today  we  are  not 
correcting  that  trend  and  it  has  led  to  the  frequent  criticism 
that  DoD  follows  a  "build  it  now,  Band-Aid  it  later"  approach 
to  acquisition.  When  we  properly  set  the  conditions  for  entry 
into  production,  we  have  achieved  high  confidence  that  we 
have  identified  and  resolved  the  major  risk  areas  and  failure 
modes,  and  will  deliver  the  needed  warfighting  capability,  not 
just  meet  contract  specs.  DT&E  is  the  means  by  which  pro¬ 
grams  determine  when  they  have  properly  set  the  conditions 
for  entry  into  production,  and  it  typically  comprises  more  than 
80  percent  of  the  T&E  activity  in  a  program  life  cycle.  However, 
more  than  80  percent  of  our  T&E  resources  in  OSD  are  allo¬ 
cated  to  oversight  of  operational  test  and  evaluation  (OT&E). 

Most  in  the  defense  T&E  community  know  that  the  DT&E 
office  in  OSD  all  but  disappeared  in  the  not-too-distant  past, 
and  that  plays  strongly  into  why  OSD  test  resources  are  so 
out  of  balance.  So  what  happened  to  DT&E  over  the  past 
40  years?  From  the  Blue  Ribbon  Defense  Panel  to  today, 
the  DoD  and  Congress  have  focused  on  OT&E.  It  is  not  un¬ 
reasonable  to  conclude  that  with  all  attention  on  OT&E,  the 
entire  acquisition  system  would  respond  accordingly  and 
shift  focus  and  resources  for  testing  to  the  right,  to  "passing 
I  OT&E"  and  getting  to  FRP.  Forty  years  of  T&E  hindsight  sug¬ 
gests  that  is  a  fundamentally  flawed  strategy.  As  the  Under 


Secretary  of  Defense  for  Acquisition,  Technology  and  Logis¬ 
tics  (USD[AT&L])  shapes  the  role  of  the  office  of  the  Deputy 
Assistant  Secretary  of  Defense  for  Developmental  Test  and 
Evaluation  (DASD[DT&E]),  history  may  be  a  valuable  tool, 
so  that  in  the  spirit  of  George  Santayana,  the  DoD  won't  be 
"condemned  to  repeat  it." 

Testing  is  a  means  to  obtain  information  to  inform  acquisition 
decisions— build  or  buy  decisions— Milestone  (MS)  decisions. 
We  need  to  think  about  how  to  improve  DT&E  to  support  ac¬ 
quisition  decisions  better.  I  provided  some  thoughts  on  how 
to  improve  and  strengthen  DT&E  in  the  "Shift  Left"  article  in 
the  September-October  2013  issue  of  Defense  AT&L  magazine. 
For  most  programs,  a  robust  DT&E  strategy  is  essential  to 
informing  the  MS  C  decision  to  enter  low-rate  initial  produc¬ 
tion  (LRIP).  If  we  don't  get  it  right  in  DT&E,  design  problems 
we  didn't  find  and  fix  before  beginning  LRIP  can  become  the 
warfighter's  problems.  By  shifting  left  to  improve  DT&E,  pro¬ 
grams  will  be  better  able  to  find  and  fix  deficiencies  before 
beginning  production,  and  that  will  lead  to  improved  acquisi¬ 
tion  outcomes. 

Follow  the  Money 

It  is  often  said  in  the  Pentagon  and  in  other  areas  of  govern¬ 
ment  that  if  you  want  to  see  how  things  get  done,  "follow  the 
money."  The  history  of  the  DT&E  office  is  clearly  visible  in  the 
funding  line. 

The  DASD(DT&E)  office  traces  its  roots  back  to  the  office  of 
the  Deputy  Director  for  Test  and  Evaluation  (DD(T&E))  cre¬ 
ated  by  David  Packard,  although  its  title  and  location  within 
the  acquisition  chain  have  changed  many  times  since  then. 


Figure  1.  Funds  for  OSD  DT&E  and  OT&E 
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In  fact,  all  of  today's  OSD  test  organizations  have  roots  in  the 
original  DD(T&E)  office,  including  the  Test  Resource  Man¬ 
agement  Center  (TRMC),  the  Foreign  Comparative  Testing 
(FCT)  Office,  and  even  the  office  of  the  Director,  Operational 
Test  and  Evaluation  (DOT&E)  and  its  subordinate  offices  for 
live-fire  and  joint  T&E  (LFT&E,  JT&E).  Throughout  those  early 
years,  the  DD(T&E)  was  responsible  for  more  than  80  percent 
of  the  OSD  test  resources.  However,  a  major  realignment  in 
June  1999  transferred  the  majority  of  resources  to  the  DOT&E 
and  virtually  eliminated  the  DT&E  office  as  an  effective  OSD 
staff  entity.  Another  10  years  would  pass,  and  the  Weapon 
Systems  Acquisition  Reform  Act  (WSARA),  Public  Law  (PL) 
111-23,  resurrected  the  DT&E  office. 

Figure  1  depicts  the  dollars  appropriated  for  the  DT&E  and 
OT&E  offices  from  fiscal  year  (FY)1973  to  the  present.  These 
are  unadjusted,  "then-year"  dollars,  precisely  as  given  in  the 
defense  appropriation  acts.  Appropriation  0450  for  the  Direc¬ 
tor,  Defense  Test  and  Evaluation  began  in  FY1973.  The  DOT&E 
position  was  created  in  1983,  but  the  first  director  was  not 
appointed  until  1985.  Thus,  funds  were  not  appropriated  for 
the  DOT&E  until  FY1987.  When  the  DOT&E  appropriation 
0460  began,  appropriation  0450  was  retitled  for  the  Direc¬ 
tor,  Developmental  Test  and  Evaluation.  The  major  shift  in  OSD 
funding  corresponds  to  the  June  1999  decision  to  transfer  T&E 
functions  to  DOT&E,  with  the  funds  moving  in  the  FY2001 
appropriation  law.  Appropriation  0450  for  the  Director  of  De¬ 
velopmental  Test  and  Evaluation  went  away  completely;  its 
programs  were  distributed  between  0460  OT&E  and  0400 
Defense  Wide  Research,  Development,  Rest  and  Evaluation 
(DW  RDT&E).  Since  there  no  longer  is  a  specific  appropriation 
for  DT&E  in  the  defense  appropriation  laws,  the  dollar  amounts 
shown  for  DT&E  from  FY2001  to  the  present  are  as  reported 
in  the  "R-1"  budget  exhibits  (see  http://comptroller.defense. 
gov).  In  FY2001,  funds  for  the  Central  Test  and  Evaluation  In¬ 
vestment  Program  (CTEIP)  moved  from  DT&E  to  OT&E,  and 
funds  for  FCT  moved  from  DT&E  into  DW  RDT&E. 

Strangely,  the  DT&E  program  element  (PE)  was  split  between 
DW  RDT&E  and  OT&E;  in  other  words,  both  the  DT&E  office 
and  the  DOT&E  were  expending  funds  assigned  to  the  same 
DT&E  PE.  The  DT&E  PE  continued  to  be  shared  until  FY2007, 
when  the  portion  of  funds  allocated  to  DOT&E  was  renum¬ 
bered  and  renamed  "operational  test  activities  and  analyses." 
The  DOT&E  also  managed  the  T&E  Science  and  Technology 
(S&T)  PE  when  it  began  in  FY2002.  The  JT&E  program  was 
transferred  to  the  DOT&E  in  December  2002  (the  money 
moved  in  FY2004),  and  in  FY2006  the  CTEIP  and  T&E  S&T 
dollars  moved  out  of  OT&E  to  DW  RDT&E  to  be  executed  by 
the  newly  created  TRMC. 

On  face  value.  Figure  1  supports  the  assertion  that  more  than 
80  percent  of  OSD  test  resources  support  OT&E  oversight. 
However,  a  significant  part  of  the  OT&E  appropriation  in¬ 
cludes  funds  for  the  LFT&E  program  and  "OT  activities  and 
analyses,"  which  now  includes  the  JT&E  program.  If  these 
are  not  considered,  what  remains  are  the  funds  for  the 


Table  1.  FY2012  Funding  for  OSD  DT&E  and 
DOT&E 


program  oversight  function.  The  imbalance  remains  large. 
For  example,  as  shown  in  Table  1,  the  FY2012  budget  (the 
most  recent  budget  unperturbed  by  sequestration)  included 
$15.8  million  for  DT&E  program  oversight  and  $60.4  million 
for  OT&E;  hence,  only  20  percent  of  the  total  $76.2  million 
funds  DT&E  program  oversight. 

How  this  resource  imbalance  came  about  is  an  interesting 
story. 

A  Brief  History  of  DT&E 
The  Original  DD(T&E) 

Deputy  Secretary  of  Defense  David  Packard  created  the  office 
of  the  DD(T&E)  in  response  to  recommendations  of  the  Blue 
Ribbon  Defense  Panel  (BRDP)  in  July  1970  (see  http://www. 
dtic.mil/dtic/tr/fulltext/u2/a013261.pdfforthe  BRDP  report). 
The  BRDP  was  essentially  concerned  about  operational  test 
and  independence.  As  described  in  the  January-February 
2014  issue  of  Defense  AT&L  magazine,  Packard  tackled  the 
BRDP  concerns  head  on.  Packard  issued  three  memoranda  in 
the  first  eight  months  of  1971  that  made  sweeping  changes  to 
the  role  of  T&E,  including  directing  the  Services  to  restructure 
their  OT&E  organizations  to  be  "separate  and  distinct  from  the 
developing  command"  and  establishing  the  DD(T&E)  within 
the  Office  of  the  Director  of  Defense  Research  and  Engineering 
(ODDR&E)  with  "across-the-board  responsibilities  for  OSD  in 
test  and  evaluation  matters."  The  ODDR&E  was  responsible 
for  major  acquisitions  at  that  time,  thus  it  preceded  both  the 
Assistant  Secretary  of  Defense  for  Research  and  Engineer¬ 
ing  that  we  know  today,  as  well  as  the  USD(AT&L).  However, 
operational  test  and  independence  would  come  to  dominate 
the  T&E  landscape. 

OT&E  and  Independence 

Many  in  DoD  had  observed  that  since  the  Director  of  De¬ 
fense  Research  and  Engineering  was  the  department's  chief 
acquisition  official,  assignment  of  the  DD(T&E)  under  this 
official  posed  a  conflict  of  interest,  and  violated  the  BRDP 
admonition  that  when  responsibilities  for  evaluation  are 
subordinated  to  the  developer,  "the  requisite  objectivity  is 
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seriously  jeopardized."  October  1977  saw  a  minor  power 
struggle  over  where  responsibilities  for  OT&E  should  be  as¬ 
signed  to  satisfy  the  BRDP  concerns,  and  it  resulted  in  re¬ 
sponsibilities  for  OT&E  being  reassigned  to  the  ASD(Program 
Analysis  and  Evaluation).  How  to  divide  the  people  and  dol¬ 
lars  proved  to  be  an  intractable  problem,  so  in  a  memo  dated 
Nov.  17, 1978,  "Operational  Test  and  Evaluation,"  Deputy  Sec¬ 
retary  Charles  W.  Duncan,  Jr.  transferred  responsibility  for 
OT&E  back  to  the  USD(R&E),  writing,  "The  Director,  Defense 
Test  and  Evaluation  is  the  cognizant  executive  for  all  Test  and 
Evaluation  matters." 

The  issue  wasn't  settled  though,  and  Congress  made  the 
next  move.  In  May  1982,  Arkansas  Sen.  David  Pryor  intro¬ 
duced  legislation  to  create  a  director  of  OT&E.  It  was  not 
well  received  in  the  Pentagon.  The  debate  about  OT&E  and 
independence  consumed  over  a  year,  and  on  Sept.  24, 1983, 


also  stipulated  that  the  DD(T&E)  would  be  redesignated  as 
Director  of  Developmental  Test  and  Evaluation. 

The  Pentagon  Wars 

In  the  early  1980s,  testing  defense  systems  became  the  cen¬ 
ter  of  attention  in  a  very  public  way  in  the  form  of  the  well- 
known  "Pentagon  Wars"  (although  the  book  did  not  come 
out  until  1993,  and  the  movie  in  1998).  Live-fire  T&E  was  the 
central  issue,  and  in  November  1986,  the  DT&E  director  cre¬ 
ated  a  new  office  to  oversee  live-fire  testing,  but  Congress 
moved  LFT&E  oversight  to  the  DOT&E  in  October  1994  (PL 
103-355  §3012). 

On  Nov.  1, 1994,  the  title  of  the  DT&E  office  changed  to  Di¬ 
rector,  Test,  Systems  Engineering  and  Evaluation  (DTSE&E). 
However,  the  pivotal  reshaping  of  DT&E  (and  DOT&E)  took 
place  in  June  1999,  prompted  by  a  pair  of  "streamlining 


By  shifting  left  to  improve  DT&E,  programs  will 
be  better  able  to  find  and  fix  deficiencies  before 
beginning  production,  and  that  will  lead  to  improved 

acquisition  outcomes. 


in  PL  98-94,  Congress  established  the  position  DOT&E,  pres- 
identially  appointed.  Senate  confirmed,  independent  of  the 
acquisition  authority  and  reporting  directly  to  the  Secretary 
of  Defense. 

The  DoD  acted  quickly  to  create  the  DOT&E  office,  although  it 
would  be20  months  before  the  DOT&Ejob  would  be  filled.  On 
Nov.  28,1983,  DASD  (Administration),  David  0.  "Doc"  Cooke, 
working  with  Richard  DeLauer,  USD(R&E),  wrote  a  memo¬ 
randum  to  the  Secretary  of  Defense,  titled  "Establishment  of 
the  Director  of  Operational  Test  and  Evaluation— ACTION 
MEMORANDUM."  Cooke  and  DeLauer  had  carefully  thought 
through  the  functions  to  be  performed  by  the  DD(T&E)  and 
the  DOT&E,  and  described  them  this  way: 

We  propose  to  adopt  a  definition  of  OT&E  which  clearly  dis¬ 
tinguishes  it  from  all  other  test  and  evaluation  activities  in  the 
development  and  acquisition  cycle.  We  recommend  that  OT&E 
apply  to  field  tests  conducted  with  production  articles  which  are 
fully  representative  of  the  intended  operational  configuration 
of  new  weapons.  All  tests  before  that  time,  whether  laboratory 
or  field,  would  be  considered  DT&E  and  part  of  the  weapon 
development  process. 

Cooke  noted  that  this  definition  was  consistent  with  congres¬ 
sional  guidance  for  ensuring  the  adequacy  of  OT&E  before 
proceeding  "beyond  low-rate  initial  production."  The  memo 


memos"  to  the  Secretary  and  Deputy  Secretary  from  Jacques 
S.  Gansler,  USD(Acquisition  &  Technology),  and  Philip  E. 
Coyle,  DOT&E,  in  May  1999.  In  the  "Plan  to  Streamline  Test 
and  Evaluation-INFORMATION  MEMORANDUM,"  Gansler 
and  Coyle  wrote: 

As  you  know,  the  Under  Secretary  of  Defense  for  Acquisition 
and  Technology  (USD[A&T])  has  advocated  for  many  years 
that  serious  testing  with  a  view  toward  operations  should  be 
started  early  in  the  life  of  a  program.  Early  testing  against  op¬ 
erational  requirements  will  provide  earlier  indications  of  mili¬ 
tary  usefulness.  It  is  also  much  less  expensive  to  correct  flaws 
in  system  design,  both  hardware  and  software,  when  caught 
early.  ...  Consistent  with  this,  the  USD(A&T)  has  decided  to 
disestablish  the  office  of  the  Director,  Test,  Systems  Engineering 
and  Evaluation  (D,TSE&E)  within  USD(A&T),  with  the  recom¬ 
mendation  to  strengthen  the  role  of  the  Director,  Operational 
Test  and  Evaluation  (DOT&E). 

Other  changes  included  transferring  oversight  of  the  major 
range  and  test  facility  base  to  DOT&E,  and  realigning  what 
remained  of  DT&E  oversight  and  the  JT&E  program  under 
the  USD(A&T)  Director  for  Strategic  and  Tactical  Systems 
(S&TS).  On  June  7, 1999,  28  years  to  the  day  after  Packard 
appointed  the  first  DD(T&E),  Secretary  of  Defense  William 
Cohen  approved  the  transfer,  and  Gansler  went  on  to  dises¬ 
tablish  DTSE&E,  and  moved  DT&E  under  S&TS. 
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Gansler  and  Coyle's  approach  is  especially  noteworthy  for  two 
reasons:  first  for  identifying  the  need  to  improve  testing— the 
authors  used  the  phrase  "serious  testing"— early  in  the  life 
cycle  and  second  for  concluding  that  the  means  to  improve 
early  testing  was  to  strengthen  the  OSD  office  that  oversees 
OT&E.  The  opportunity  to  shift  focus  toward  "serious"  devel¬ 
opmental  testing  slipped  through  their  grasp,  and  it  ushered 
in  a  decade  of  declining  interest  in  OSD  DT&E  oversight  in 
particular,  and  program  DT&E  in  general. 

The  erosion  of  DT&E  responsibilities  continued,  and  on  Dec. 
9,  2002,  the  USD(AT&L)  transferred  the  JT&E  program 
to  the  DOT&E.  What  remained  of  the  DT&E  organization 
moved  again,  this  time  placed  under  the  director.  Systems 
and  Software  Engineering.  Finally,  on  May  22,  2009,  the 
WSARA,  PL  111-23  Section  102,  reversed  the  decline  and 
re-established  the  DT&E  office.  Now  in  the  post-WSARA 
era,  we  have  an  opportunity  to  change  course,  to  shift  focus 
to  DT&E  and  readiness  for  production;  we  must  not  let  it 
slip  through  our  grasp. 

Conclusion 

To  the  question  posed  in  the  title  of  this  article— "What  Hap¬ 
pened  to  DT&E?"— I  submit  that  the  circumstances  that  de¬ 
cades  ago  prompted  creation  of  the  Operational  Test  Agen¬ 
cies  and  DOT&E  caused  us  to  take  our  eyes  off  the  target. 
The  breadth  of  DT&E  was  understood  by  Cooke  and  DeLauer 
when  they  proposed  that  "OT&E  apply  to  field  tests  conducted 


with  production  articles  which  are  fully  representative  of  the 
intended  operational  configuration  of  new  weapons.  All  tests 
before  that  time,  whether  laboratory  or  field,  would  be  con¬ 
sidered  DT&E  and  part  of  the  weapon  development  process." 
However,  instead  of  building  and  resourcing  an  organization 
to  oversee  the  magnitude  of  developmental  testing  that  state¬ 
ment  describes,  the  department  put  its  priorities  on  OT&E. 
In  the  post-WSARA  era,  each  major  defense  acquisition 
program  is  to  have  a  chief  developmental  tester  and  a  gov¬ 
ernment  organization  serve  as  lead  DT&E  organization.  The 
chief  developmental  tester  and  lead  DT&E  organization  must 
assume  responsibility  for  planning  and  conducting  robust 
DT&E  in  a  mission  context— or  using  words  from  the  past, 
"serious  testing  with  a  view  toward  operations  early  in  the  life 
of  a  program"—  to  identify  risks,  correct  deficiencies,  and  set 
the  conditions  for  entry  into  production.  Developmental  test 
and  evaluation  is  the  key  to  improving  acquisition  outcomes. 

Note:  The  author  would  like  to  thank  the  following  for  contributing  to  the 
history  project:  Jack  Krings,  Pete  Adolph,  Tom  Christie,  Joe  Navarro,  Steve 
Kimmel,  Irv  Boyles,  Charlie  Ackerman,  John  Bolino,  Pat  Sanders,  Charles 
Watt,  Jim  O'Bryon,  Rick  Lockhart,  Chris  DiPetto,  Rich  Stuckey,  Parker  Horner, 
the  OSD  Historian's  Office,  and  the  OSD  Correspondence  Office.  I  apologize 
to  anyone  I  have  inadvertently  left  off  this  list.  Finally,  I  want  to  offer  special 
thanks  to  Stephanie  Lindemann  in  ODASD(DT&E)  for  her  outstanding  re¬ 
search  and  assistance. 


The  author  can  be  contacted  at  steven.hutchison@hq.dhs.gov. 


Where  Can  You  Get 
the  Latest  on  the 

Better  Buying  Power 

Initiatives? 


■  BBP  Gateway  (https://dap.dau.mil/bbp)  is  your  source  for  the 
latest  information,  guidance,  and  directives  on  better  buying 
power  in  defense  acquisition 


■  BBP  Public  Site  (https://acc.dau.mil/bbp)  is  your  forum  to  share 
BBP  knowledge  and  experience 
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